We have a system programing on spark and oracle DB. the bottleneck of my system is when workers would insert or update in DB, there are timeout error on some executors of workers. We have 6 worker which has 64G RAM and 8 cores. number of execution is equal to cores of worker and each executor run a task. oracle DB has 16 cores and 96G RAM. We think our spark cluster is bigger than DB system and insert or update for big data(each executor maybe insert or update 7G data simultaneously) is it true?
the other issue is that the distributed DB like Cassandra is a solution for this bottleneck?